Generating Synthetic Labeled Data From Existing Anatomical Models: An Example With Echocardiography Segmentation.

Deep learning can bring time savings and increased reproducibility to medical image analysis. However, acquiring training data is challenging due to the time-intensive nature of labeling and high inter-observer variability in annotations. Rather than labeling images, in this work we propose an alternative pipeline where images are generated from existing high-quality annotations using generative adversarial networks (GANs). Annotations are derived automatically from previously built anatomical models and are transformed into realistic synthetic ultrasound images with paired labels using a CycleGAN. We demonstrate the pipeline by generating synthetic 2D echocardiography images to compare with existing deep learning ultrasound segmentation datasets. A convolutional neural network is trained to segment the left ventricle and left atrium using only synthetic images. Networks trained with synthetic images were extensively tested on four different unseen datasets of real images with median Dice scores of 91, 90, 88, and 87 for left ventricle segmentation. These results match or are better than inter-observer results measured on real ultrasound datasets and are comparable to a network trained on a separate set of real images. Results demonstrate the images produced can effectively be used in place of real data for training. The proposed pipeline opens the door for automatic generation of training data for many tasks in medical imaging as the same process can be applied to other segmentation or landmark detection tasks in any modality. The source code and anatomical models are available to other researchers.1 1https://adgilbert.github.io/data-generation/.

User-Intended Doppler Measurement Type Prediction Combining CNNs With Smart Post-Processing.

Spectral Doppler measurements are an important part of the standard echocardiographic examination. These measurements give insight into myocardial motion and blood flow, providing clinicians with parameters for diagnostic decision making. Many of these measurements are performed automatically with high accuracy, increasing the efficiency of the diagnostic pipeline. However, full automation is not yet available because the user must manually select which measurement should be performed on each image. In this work, we develop a pipeline based on convolutional neural networks (CNNs) to automatically classify the measurement type from cardiac Doppler scans. We show how the multi-modal information in each spectral Doppler recording can be combined using a meta parameter post-processing mapping scheme and heatmaps to encode coordinate locations. Additionally, we experiment with several architectures to examine the tradeoff between accuracy, speed, and memory usage for resource-constrained environments. Finally, we propose a confidence metric using the values in the last fully connected layer of the network and show that our confidence metric can prevent many misclassifications. Our algorithm enables a fully automatic pipeline from acquisition to Doppler spectrum measurements. We achieve 96% accuracy on a test set drawn from separate clinical sites, indicating that the proposed method is suitable for clinical adoption.

The 'Digital Twin' to enable the vision of precision cardiology.

Providing therapies tailored to each patient is the vision of precision medicine, enabled by the increasing ability to capture extensive data about individual patients. In this position paper, we argue that the second enabling pillar towards this vision is the increasing power of computers and algorithms to learn, reason, and build the 'digital twin' of a patient. Computational models are boosting the capacity to draw diagnosis and prognosis, and future treatments will be tailored not only to current health status and data, but also to an accurate projection of the pathways to restore health by model predictions. The early steps of the digital twin in the area of cardiovascular medicine are reviewed in this article, together with a discussion of the challenges and opportunities ahead. We emphasize the synergies between mechanistic and statistical models in accelerating cardiovascular research and enabling the vision of precision medicine.

Cardiac Motion Evolution Model for Analysis of Functional Changes Using Tensor Decomposition and Cross-Sectional Data.

Cardiac disease can reduce the ability of the ventricles to function well enough to sustain long-term pumping efficiency. Recent advances in cardiac motion tracking have led to improvements in the analysis of cardiac function. We propose a method to study cohort effects related to age with respect to cardiac function. The proposed approach makes use of a recent method for describing cardiac motion of a given subject using a polyaffine model, which gives a compact parameterization that reliably and accurately describes the cardiac motion across populations. Using this method, a data tensor of motion parameters is extracted for a given population. The partial least squares method for higher order arrays is used to build a model to describe the motion parameters with respect to age, from which a model of motion given age is derived. Based on the cross-sectional statistical analysis with the data tensor of each subject treated as an observation along time, the left ventricular motion over time of Tetralogy of Fallot patients is analysed to understand the temporal evolution of functional abnormalities in this population compared to healthy motion dynamics.

Detecting Clinically Meaningful Shape Clusters in Medical Image Data: Metrics Analysis for Hierarchical Clustering Applied to Healthy and Pathological Aortic Arches.

OBJECTIVE: Today's growing medical image databases call for novel processing tools to structure the bulk of data and extract clinically relevant information. Unsupervised hierarchical clustering may reveal clusters within anatomical shape data of patient populations as required for modern precision medicine strategies. Few studies have applied hierarchical clustering techniques to three-dimensional patient shape data and results depend heavily on the chosen clustering distance metrics and linkage functions. In this study, we sought to assess clustering classification performance of various distance/linkage combinations and of different types of input data to obtain clinically meaningful shape clusters. METHODS: We present a processing pipeline combining automatic segmentation, statistical shape modeling, and agglomerative hierarchical clustering to automatically subdivide a set of 60 aortic arch anatomical models into healthy controls, two groups affected by congenital heart disease, and their respective subgroups as defined by clinical diagnosis. Results were compared with traditional morphometrics and principal component analysis of shape features. RESULTS: Our pipeline achieved automatic division of input shape data according to primary clinical diagnosis with high F-score (0.902 ± 0.042) and Matthews correlation coefficient (0.851 ± 0.064) using the correlation/weighted distance/linkage combination. Meaningful subgroups within the three patient groups were obtained and benchmark scores for automatic segmentation and classification performance are reported. CONCLUSION: Clustering results vary depending on the distance/linkage combination used to divide the data. Yet, clinically relevant shape clusters and subgroups could be found with high specificity and low misclassification rates. SIGNIFICANCE: Detecting disease-specific clusters within medical image data could improve image-based risk assessment, treatment planning, and medical device development in complex disease.

Looks Do Matter! Aortic Arch Shape After Hypoplastic Left Heart Syndrome Palliation Correlates With Cavopulmonary Outcomes.

BACKGROUND: Aortic arch reconstruction after hypoplastic left heart syndrome (HLHS) palliation can vary widely in shape and dimensions between patients. Arch morphology alone may affect cardiac function and outcome. We sought to uncover the relationship of arch three-dimensional shape features with functional and short-term outcome data after total cavopulmonary connection (TCPC). METHODS: Aortic arch shape models of 37 patients with HLHS (age, 2.89 ± 0.99 years) were reconstructed from magnetic resonance data before TCPC completion. A novel, validated statistical shape analysis method was used to compute a three-dimensional anatomic mean shape from the cohort and calculate the deformation vectors of the mean shape toward each patient's specific anatomy. From these deformations, three-dimensional shape features most related to ventricular ejection fraction, indexed end-diastolic volume, and superior cavopulmonary pressure were extracted by partial least-square regression analysis. Shape patterns relating to intensive care unit and hospital lengths of stay after TCPC were assessed. RESULTS: Distinct deformation patterns, which result in an acutely mismatched aortic root and ascending aorta, and a gothic-like transverse arch, correlated with increased indexed end-diastolic volume and higher superior cavopulmonary pressure but not with ejection fraction. Specific arch morphology with pronounced transverse arch and descending aorta mismatch also correlated with longer intensive care unit and hospital lengths of stay after TCPC completion. CONCLUSIONS: Independent of hemodynamically important arch obstruction, altered aortic morphology in HLHS patients appears to have important associations with higher superior cavopulmonary pressure and with short-term outcomes after TCPC completion as highlighted by statistical shape analysis, which could act as adjunct to risk assessment in HLHS.

How successful is successful? Aortic arch shape after successful aortic coarctation repair correlates with left ventricular function.

OBJECTIVES: Even after successful aortic coarctation repair, there remains a significant incidence of late systemic hypertension and other morbidities. Independently of residual obstruction, aortic arch morphology alone may affect cardiac function and outcome. We sought to uncover the relationship of arch 3-dimensional shape features with functional data obtained from cardiac magnetic resonance scans. METHODS: Three-dimensional aortic arch shape models of 53 patients (mean age, 22.3 ± 5.6 years) 12 to 38 years after aortic coarctation repair were reconstructed from cardiac magnetic resonance data. A novel validated statistical shape analysis method computed a 3-dimensional mean anatomic shape of all aortic arches and calculated deformation vectors of the mean shape toward each patient's arch anatomy. From these deformations, 3-dimensional shape features most related to left ventricular ejection fraction, indexed left ventricular end-diastolic volume, indexed left ventricular mass, and resting systolic blood pressure were extracted from the deformation vectors via partial least-squares regression. RESULTS: Distinct arch shape features correlated significantly with left ventricular ejection fraction (r = 0.42, P = .024), indexed left ventricular end-diastolic volume (r = 0.65, P < .001), and indexed left ventricular mass (r = 0.44, P = .014). Lower left ventricular ejection fraction, larger indexed left ventricular end-diastolic volume, and increased indexed left ventricular mass were identified with an aortic arch shape that has an elongated ascending aorta with a high arch height-to-width ratio, a relatively short proximal transverse arch, and a relatively dilated descending aorta. High blood pressure seemed to be linked to gothic arch shape features, but this did not achieve statistical significance. CONCLUSIONS: Independently of hemodynamically important arch obstruction or residual aortic coarctation, specific aortic arch shape features late after successful aortic coarctation repair seem to be associated with worse left ventricular function. Analyzing 3-dimensional shape information via statistical shape modeling can be an adjunct to long-term risk assessment in patients after aortic coarctation repair.

Ventricular structure in ARVC: going beyond volumes as a measure of risk.

BACKGROUND: Altered right ventricular structure is an important feature of Arrhythmogenic Right Ventricular Cardiomyopathy (ARVC), but is challenging to quantify objectively. The aim of this study was to go beyond ventricular volumes and diameters and to explore if the shape of the right and left ventricles could be assessed and related to clinical measures. We used quantifiable computational methods to automatically identify and analyse malformations in ARVC patients from Cardiovascular Magnetic Resonance (CMR) images. Furthermore, we investigated how automatically extracted structural features were related to arrhythmic events. METHODS: A retrospective cross-sectional feasibility study was performed on CMR short axis cine images of 27 ARVC patients and 21 ageing asymptomatic control subjects. All images were segmented at the end-diastolic (ED) and end-systolic (ES) phases of the cardiac cycle to create three-dimensional (3D) bi-ventricle shape models for each subject. The most common components to single- and bi-ventricular shape in the ARVC population were identified and compared to those obtained from the control group. The correlations were calculated between identified ARVC shapes and parameters from the 2010 Task Force Criteria, in addition to clinical outcomes such as ventricular arrhythmias. RESULTS: Bi-ventricle shape for the ARVC population showed, as ordered by prevalence with the percent of total variance in the population explained by each shape: global dilation/shrinking of both ventricles (44 %), elongation/shortening at the right ventricle (RV) outflow tract (15 %), tilting at the septum (10 %), shortening/lengthening of both ventricles (7 %), and bulging/shortening at both the RV inflow and outflow (5 %). Bi-ventricle shapes were significantly correlated to several clinical diagnostic parameters and outcomes, including (but not limited to) correlations between global dilation and electrocardiography (ECG) major criteria (p = 0.002), and base-to-apex lengthening and history of arrhythmias (p = 0.003). Classification of ARVC vs. control using shape modes yielded high sensitivity (96 %) and moderate specificity (81 %). CONCLUSION: We presented for the first time an automatic method for quantifying and analysing ventricular shapes in ARVC patients from CMR images. Specific ventricular shape features were highly correlated with diagnostic indices in ARVC patients and yielded high classification sensitivity. Ventricular shape analysis may be a novel approach to classify ARVC disease, and may be used in diagnosis and in risk stratification for ventricular arrhythmias.

A statistical shape modelling framework to extract 3D shape biomarkers from medical imaging data: assessing arch morphology of repaired coarctation of the aorta.

BACKGROUND: Medical image analysis in clinical practice is commonly carried out on 2D image data, without fully exploiting the detailed 3D anatomical information that is provided by modern non-invasive medical imaging techniques. In this paper, a statistical shape analysis method is presented, which enables the extraction of 3D anatomical shape features from cardiovascular magnetic resonance (CMR) image data, with no need for manual landmarking. The method was applied to repaired aortic coarctation arches that present complex shapes, with the aim of capturing shape features as biomarkers of potential functional relevance. The method is presented from the user-perspective and is evaluated by comparing results with traditional morphometric measurements. METHODS: Steps required to set up the statistical shape modelling analyses, from pre-processing of the CMR images to parameter setting and strategies to account for size differences and outliers, are described in detail. The anatomical mean shape of 20 aortic arches post-aortic coarctation repair (CoA) was computed based on surface models reconstructed from CMR data. By analysing transformations that deform the mean shape towards each of the individual patient's anatomy, shape patterns related to differences in body surface area (BSA) and ejection fraction (EF) were extracted. The resulting shape vectors, describing shape features in 3D, were compared with traditionally measured 2D and 3D morphometric parameters. RESULTS: The computed 3D mean shape was close to population mean values of geometric shape descriptors and visually integrated characteristic shape features associated with our population of CoA shapes. After removing size effects due to differences in body surface area (BSA) between patients, distinct 3D shape features of the aortic arch correlated significantly with EF (r = 0.521, p = .022) and were well in agreement with trends as shown by traditional shape descriptors. CONCLUSIONS: The suggested method has the potential to discover previously unknown 3D shape biomarkers from medical imaging data. Thus, it could contribute to improving diagnosis and risk stratification in complex cardiac disease.

Spatio-Temporal Tensor Decomposition of a Polyaffine Motion Model for a Better Analysis of Pathological Left Ventricular Dynamics.

Given that heart disease can cause abnormal motion dynamics over the cardiac cycle, understanding and quantifying cardiac motion can provide insight for clinicians to aid with diagnosis, therapy planning, and determining prognosis. The goal of this paper is to extract population-specific motion patterns from 3D displacements in order to identify the mean motion in a population, and to describe pathology-specific motion patterns in terms of the spatial and temporal components. Since there are common motion patterns observed in patients with the same condition, extracting these can lead towards a better understanding of the disease. Quantifying cardiac motion at a population level is not a simple task since images can vary widely in terms of image quality, size, resolution, and pose. To overcome this, we analyze the parameters obtained from a cardiac-specific Polyaffine motion-tracking algorithm, which are aligned both spatially and temporally to a common reference space. Once all parameters are aligned, different subjects can be compared and analyzed in the space of Polyaffine transformations by projecting the transformations to a reduced order subspace in which dominant motion patterns in each population can be extracted. Using tensor decomposition, the spatial and temporal aspects can be decoupled in order to study the components individually. The proposed method was validated on healthy volunteers and Tetralogy of Fallot patients according to known spatial and temporal behavior for each population. A key advantage of this method is the ability to regenerate motion sequences from the models, which can be visualized in terms of the full motion.

Group-wise construction of reduced models for understanding and characterization of pulmonary blood flows from medical images.

3D computational fluid dynamics (CFD) in patient-specific geometries provides complementary insights to clinical imaging, to better understand how heart disease, and the side effects of treating heart disease, affect and are affected by hemodynamics. This information can be useful in treatment planning for designing artificial devices that are subject to stress and pressure from blood flow. Yet, these simulations remain relatively costly within a clinical context. The aim of this work is to reduce the complexity of patient-specific simulations by combining image analysis, computational fluid dynamics and model order reduction techniques. The proposed method makes use of a reference geometry estimated as an average of the population, within an efficient statistical framework based on the currents representation of shapes. Snapshots of blood flow simulations performed in the reference geometry are used to build a POD (Proper Orthogonal Decomposition) basis, which can then be mapped on new patients to perform reduced order blood flow simulations with patient specific boundary conditions. This approach is applied to a data-set of 17 tetralogy of Fallot patients to simulate blood flow through the pulmonary artery under normal (healthy or synthetic valves with almost no backflow) and pathological (leaky or absent valve with backflow) conditions to better understand the impact of regurgitated blood on pressure and velocity at the outflow tracts. The model reduction approach is further tested by performing patient simulations under exercise and varying degrees of pathophysiological conditions based on reduction of reference solutions (rest and medium backflow conditions respectively).